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METHOD AND APPARATUS FOR DEI NTERLAC ING 
FIELD OF THE INVENTION 

The present invention relates to a deinterlacing method and 
deinterlacing apparatus for converting an interlaced image signal 
into a progressive image signal. 

BACKGROUND OF THE INVENTION 

In recent years , digital processing for images has been 
coming into wide use. As a method for compressively coding image 
data, high-efficiency coding such as MPEG (Moving Picture Experts 
Group) is adopted. The high-efficiency coding is a technique for 
coding image data at a low bit rate to increase the efficiency 
of digital transmission, recording and the like. 

In the current NTSC television broadcasts, an interlaced 
scanning method is adopted. By the interlaced scanning method, 
one screen (one frame) is divided into a top field and a bottom 
field and transmitted, thereby saving the bandwidth and enabling 
a high-efficiency transmission. However, by the interlaced 
scanning method, line flicker and line crawling is conspicuous 
due to the high intensity and enlarged screen of the television 
receiver. Therefore, in some cases, the deinterlacing for 
converting an interlaced image signal into a progressive image 
signal is performed by performing the interpolation using an image 
memory. 

Japanese Published Patent Application No. Hei. 7-131761 
(Document 1) discloses an example of the deinterlacing method for 
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converting an interlaced image signal into a progressive image 
signal. This method is called also a VT filtering method. By this 
VT filtering method, pixels in three successive fields, i.e., a 
field for which an interpolation pixel is generated and forward 
and backward fields are subjected to a filtering process, to 
generate the interpolation pixel, and an interlaced image is 
converted into a progressive image using the generated 
interpolation pixel. Hereinafter, this VT filtering method will 
be described with reference to figure 25. 

Figure 25 is a diagram schematically showing a state of an 
interlaced image on a time-vertical plane, in which a white dot 
shows a pixel (scan line). Here, the description is given of a 
filtering process in a case where a pixel in a position "k' " shown 
in figure 25 is generated to perform interpolation. 

In this case, by the VT filtering method, pixels of three 
successive fields, i.e., a field including the position ff k'" and 
forward and backward fields, for example 10 pixels "a' j ' " as 
shown in figure 25, are subjected to the filtering process , thereby 
generating the pixel of the position "k' " . At that time, a filter 
having vertical low-pass characteristics for pixels in the same 
field (pixels »d' "-"g"' ) and vertical high-pass characteristics 
for pixels in adjacent fields (pixels »a' ,, ~ ,, c ,,, and "h' j ' " ) 
is used in the filtering process. The filter having such 
characteristics extracts vertical low-pass components from the 
pixels "d' »-"g' ,f and vertical high-pass components from the pixels 
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»a'»~»c'" and «h' j ' " . By the VT filtering method, the low- 
pass components and high-pass components extracted by the filter 
are added to generate the interpolation pixel* 

Japanese Published Patent Application No. Hei. 11-331782 
(Document 2) discloses another example of the deinterlacing method. 
The method disclosed in this application is a method for 
implementing the deinterlacing method by a signal conversion 
apparatus having a still picture filter, a moving picture filter, 
a motion detection circuit and a mixer. According to this method, 
initially, the still picture filter and the motion picture filter 
generate an interpolation pixel, respectively. Next, on the basis 
of a result of detection by the motion detection circuit for 
detecting the motion of an image, the mixer mixes the interpolation 
signals outputted from the still picture filter and the moving 
picture filter, thereby generating a final interpolation pixel. 
Then, using the final interpolation pixel, the interlaced image 
is converted into a progressive image. Here, the motion detection 
circuit calculates a difference between frames and obtains the 
quantity of motion from the absolute value of the difference. 

Hereinafter, this deinterlacing method will be described 
with reference to figure 25. Here, the description is given of 
the filtering process in a case where a pixel in a position "k /M 
shown in figure 25 is generated to perform interpolation. 

According to this method, initially, the still picture 
filter subjects the pixels "b'", "e ,,f , "f" and "i ,n to the 
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filtering process to generate an interpolation pixel M k'", and 
the moving picture filter subjects the pixels »a / "~"j'" to the 
filtering process to generate an interpolation pixel "k' " . Here, 
the filter coefficient of the still picture filter is set so that 
weights to the pixels "b'" and "i' n are the most heavy, and the 
filter coefficient of the moving picture filter is set so that 
weights to the pixels "d'", "e' n , "f" and"g'" are heavier. Next, 
the motion detection circuit detects a motion of the image from 
a difference between frames, and on the basis of the detection 
result, the mixer mixes the interpolation signal outputted from 
the still picture filter and the interpolation signal outputted 
from the moving picture filter, thereby generating the final 
interpolation pixel. 

In the above description, the filtering process is performed 
for pixels in three fields, while the filtering process may be 
performed for pixels in two fields. In that case, the still 
picture filter subjects the pixels "e"', "f M ' and "i"' to the 
filtering process and the moving picture filter subjects the 
pixels «d'"~" j'" to the filtering process, thereby generating the 
interpolation pixel "k'" . 

As another example of the deinterlacing method, there is 
a method which detects a motion vector between a field which is 
subjected to the deinterlacing and its adjacent field, obtains 
an interpolation pixel from the adjacent field on the basis of 
the detection result of the motion vector, and converts an 
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interlaced image into a progressive image using the interpolation 
pixel. According to this method, since the motion of the image 
is detected as the motion vector information, the interpolation 
process which almost matches the motion of the image can be 
performed, while the detection of the motion vector requires a 
massive quantity of signal processing. 

Thus, Japanese Published Patent Application No, Hei.10- 
126749 (Document 3) discloses a deinterlacing apparatus that 
implements a method for performing a deinterlacing using a motion 
vector, in which the quantity of processing for detecting the 
motion vector is reduced. 

This deinterlacing apparatus disclosed in this application 
receives an interlaced image signal (MPEG video) that is coded 
by a coding method such as MPEG, and adaptively changing the 
interpolation method in the deinterlacing by utilizing 
information such as a structure of an image included in the MPEG 
video and a motion vector, whereby circuits such as a motion 
detection circuit and a motion vector detection circuit are 
omitted. Here, the description is given of a case where the MPG 
video inputted to the deinterlacing apparatus is coded by a MPEG2 
coding method. 

MPEG2 coding process includes an intra-frame coding and an 
inter-frame coding. When the intra-frame coding is performed to 
an interlaced image, the interlaced image is subjected a DCT 
(Discrete Cosine Transformation) process for example in units of 
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block { DCT block) being composed of 8 x 8 pixels, its spatial 
coordinate components are converted into frequency components, 
and thereafter a variable-length coding is performed. 

On the other hand, when the inter-frame coding is performed 
to the interlaced image, differences between the image of the 
present frame and reference images of forward and backward frames 
are obtained as predicted errors, and the predicted errors are 
subjected to a DCT process, a quantization process and a 
variable-length coding process. Accordingly, the quantity of 
codes can be significantly decreased. However, in a case where 
the motion of the image is large, when the differences between 
the present image and the forward and backward images are simply 
obtained, the predicted errors may be large and therefore the 
quantity of codes may be adversely increased. Then, motion 
vectors between the reference images and the image of the present 
frame are detected, and the reference images are motion- 
compensated on the basis of the motion vectors to obtain the 
differences from the image of the present frame, whereby the 
predicted errors are reduced to decrease the quantity of codes. 
Here, in the MPEG2 coding, one image (picture) which consists of 
one frame or one field is divided into macroblocks, each being 
composed of 16 lines x 16 pixels, and then the motion vector is 
detected in units of macroblock, thereby performing the motion 
compensation . 

Among images which have been coded in the above-mentioned 



coding process, an intra-f rame coded image that is obtained by 
predictive-coding image data in one frame is called I picture, 
an inter-frame forward predictive-coded image that is obtained 
by coding image data by a predictive coding using a forward frame 
is called P picture, and a bidirectionally predictive-coded image 
that is obtained by coding image data by a predictive coding using 
any of a forward frame, a backward frame, and bidirectional frames 
is called B picture. 

The coding of the interlaced image is performed in following 
cases: a case where images of top and bottom fields are directly 
coded, i.e. , the coding is performed in states of field structure, 
and a case where a frame image is made from top and bottom field 
images and the coding is performed in a state of frame structure. 
In the coding that is performed in the state of field structure, 
the motion compensation prediction and the DCT coding are 
performed in units of field picture. On the other hand, in the 
coding in the state of frame structure, the coding is performed 
in units of frame picture, and frame prediction which uses a frame 
image as a reference image and field prediction which uses one 
of the top field image and the bottom field image as a reference 
image are adopted as the motion compensation prediction. 

Therefore, by decoding a code sequence of MPEG video which 
is coded as described above, the deinterlacing apparatus can 
obtain information such as the image structure at the coding of 
an interlaced image, the motion vector and the prediction mode. 
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Then, the apparatus judges whether the decoded image data is a 
static area or a moving area on the basis of the obtained 
information. When the apparatus judges that the image data is a 
static area, it performs the inter-field interpolation , and when 
it judges that the image data is a moving area, it performs the 
intra-field interpolation, thereby converting the interlaced 
image signal into a progressive image signal. 

However, in the deinterlacing, the vertical resolution of 
the progressive image can be increased more in the case where the 
inter-field interpolation is employed, as compared to the case 
where the intra-field interpolation is employed. Therefore, even 
when the decoded image data is a moving area, the deinterlacing 
apparatus judges whether the inter-field interpolation can be 
performed or not. Here, the quantity of motion of the motion 
vector at the coding included in the MPEG video is calculated, 
and when the quantity of motion is equivalent to even-numbered 
pixels, it is judged that the inter-field interpolation can be 
performed. Then, when it is judged that the inter-field 
interpolation can be performed even in a moving area, the 
inter-field interpolation is performed on the basis of the motion 
vector, whereby the vertical resolution of the converted 
progressive image can be increased. 

However, according to the VT filtering method described in 
Document 1, the interpolation pixel is generated regardless of 
whether the interlaced image is a still picture or a moving picture. 
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That is, the filtering process is always performed to pixels of 
three successive fields, i.e., a field for which an interpolation 
pixel is generated and forward and backward fields. Thus, in the 
case of still picture, a high-resolution progressive image can 
be generated. On the other hand, in the case of moving picture, 
the correlation between the field for which the interpolation 
pixel is generated and the forward and backward fields is low, 
so that when pixels of the forward and backward fields are subjected 
to the filtering process to generate the interpolation pixel, 
degradation of the image quality may occur. Particularly in a case 
where video including an oblique edge moves vertically or the like, 
degradation occurs, for example the edge looks jagged. 

In addition, according to the method described in Document 
2, two kinds of filters, i.e., the still picture filter and the 
moving picture filter are required for generating an interpolation 
pixel, so that the circuit scale is adversely increased. 

Further, generally, according to the MPEG2 coding method, 
when an interlaced image is motion-compensated and then 
predictive-coded, a motion vector between frames which are one 
to three frames away from each other is detected, and a reference 
image is motion-compensated using the motion vector. Therefore, 
when the deinterlacing apparatus performs the inter-field 
interpolation by directly using the motion vector that is obtained 
at the decoding of MPEG video, an interpolation pixel may be 
obtained from a frame that is several frames apart from the target 



-10- 

frame. When a motion vector is used at the motion compensation 
in the coding, even when its detection accuracy is low, only the 
coding efficiency is slightly worsened and the image quality is 
not remarkably degraded. Therefore, it is no problem when the 
interpolation pixel is obtained from a frame that is several frames 
apart from the target frame. However, in the deinterlacing that 
performs interpolation of pixels on the basis of a motion vector, 
the motion vector should be detected at higher accuracy. 
Therefore, in the deinterlacing apparatus described in Document 
3, when an interpolation pixel is obtained by directly using a 
motion vector at the motion compensation in the coding, the 
interpolation pixel may be obtained from a frame that is several 
frames apart from the target frame, and thus an erroneous 
interpolation pixel may be generated. 

SUMMARY OF THE INVENTION 

The present invention has for its object to provide a 
deinterlacing method and a deinterlacing apparatus, which can 
generate a progressive image of a high solution for a still picture 
and can generate progressive image having no degradation of the 
image quality for a moving picture, with a small scale of a circuit 
structure. 

It is another object of the present invention is to provide 
a deinterlacing method and a deinterlacing apparatus, which can 
generate an interpolation pixel at high accuracy, even when the 
deinterlacing is performed using a motion vector that is obtained 



by decoding a code sequence of MPEG video* 

Other objects and advantages of the present invention will 
become apparent from the detailed description and specific 
embodiments described are provided only for illustration since 
various additions and modifications within the spirit and scope 
of the invention will be apparent to those of skill in the art 
from the detailed description. 

According to a 1st aspect of the present invention, there 
is provided a deinterlacing method for converting an interlaced 
image into a progressive image, comprising steps of: performing 
a filtering process to pixels of at least one of three fields, 
a deinterlacing target field to be subjected to a deinterlacing 
process and forward and backward fields of the deinterlacing 
target field within the interlaced image, thereby generating an 
interpolation pixel for the deinterlacing target field; measuring 
a quantity of motion of the deinterlacing target field; and 
changing characteristics of the filtering on the basis of the 
quantity of the motion. 

According to a 2nd aspect of the present invention, in the 
deinterlacing method of the 1st aspect, a filter which is used 
for the filtering process in the step of generating the 
interpolation pixel has characteristics of extracting vertical 
low frequency components of the deinterlacing target field, and 
extracting vertical high frequency components of the forward and 
backward fields of the deinterlacing target field. 
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According to a 3rd aspect of the present invention, in the 
deinterlacing method of the 1st aspect, in the step of generating 
the interpolation pixel, pixels in the deinterlacing target field 
or peripheral fields, which are in the same horizontal position 
as that of a position to be interpolated are subjected to the 
filtering process. 

According to a 4th aspect of the present invention, in the 
deinterlacing method of the 1st aspect, in the step of measuring 
the quantity of the motion, the quantity of the motion is obtained 
from a difference between the deinterlacing target field or a frame 
including the deinterlacing target field, and other field or 
frame. 

According to a 5th aspect of the present invention, in the 
deinterlacing method of the 1st aspect, in the step of measuring 
the quantity of the motion, the quantity of the motion is obtained 
from a difference between the pixels which are used when the 
filtering process is performed in the step of generating the 
interpolation pixels. 

According to a 6th aspect of the present invention, in the 
deinterlacing method of the 5th aspect, in the step of measuring 
the quantity of the motion, the quantity of the motion is obtained 
from a difference between pixels which are included in the forward 
and backward fields of the deinterlacing target field, among the 
pixels which are used when the filtering process is performed in 
the step of generating the interpolation pixels. 



According to a 7th aspect of the present invention, in the 
deinterlacing method of the 1st aspect, in the step of changing 
characteristics of the filtering, the characteristics of the 
filtering are changed so that gain of components from the forward 
and backward fields of the deinterlacing target field is reduced 
as the quantity of the motion is increased. 

According to an 8th aspect of the present invention, in the 
deinterlacing method of the 1st aspect, in the step of changing 
characteristics of the filtering, the characteristics of the 
filtering are changed so that gain of components from the forward 
and backward fields of the deinterlacing target field is reduced 
to zero when the quantity of the motion is large* 

According to a 9th aspect of the present invention, there 
is provided a deinterlacing apparatus for converting an interlaced 
image into a progressive image, comprising: a frame memory for 
storing the interlaced image; a filter unit for receiving a 
deinterlacing target field to be subjected to a deinterlacing 
process and one or both of forward and backward fields of the 
deinterlacing target field within the interlaced image, from the 
frame memory, and performing a filtering process to pixels of at 
least one of the received fields, thereby generating an 
interpolation pixel for the deinterlacing target field; a 
difference operation unit for measuring a quantity of motion of 
the deinterlacing target field; and a filter coefficient setting 
unit for changing characteristics of the filter unit on the basis 



of the quantity of the motion measured by the difference operation 
unit. 

According to a 10th aspect of the present invention, there 
is provided a deinterlacing apparatus for converting an interlaced 
image into a progressive image comprising: a frame memory for 
storing the interlaced image; a filter unit for receiving a 
deinterlacing target field to be subjected to a deinterlacing 
process and one or both of forward and backward fields of the 
deinterlacing target field within the interlaced image, from the 
frame memory, and performing a filtering process to pixels of at 
least one of the received fields, thereby generating an 
interpolation pixel for the interlacing target field; a difference 
operation unit for receiving the deinterlacing target field or 
a frame including the deinterlacing target field, and a field or 
frame which is adjacent to the deinterlacing target field or frame 
including the deinterlacing target field within the interlaced 
image, from the frame memory, and operating a difference 
therebetween, thereby measuring a quantity of motion of the 
deinterlacing target field; a filter coefficient setting unit for 
changing filter characteristics of the filter unit on the basis 
of the quantity of the motion measured by the difference operation 
unit; and a double-speed converter for composing the interlaced 
image and the interpolation pixel generated by the filter unit, 
and generating the progressive image. 

According to an 11th aspect of the present invention, there 
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is provided a deinterlacing method for performing a decoding 
process to a code sequence, field by field or frame by frame, which 
code sequence is obtained by coding an interlaced image composed 
of plural fields using motion compensation, and converting a 
decoded image of the interlaced image, obtained by the decoding 
process, into a progressive image, comprising: a decoding step 
of decoding the interlaced image, thereby obtaining the decoded 
image as well as obtaining a motion vector at the motion 
compensation which indicates a prescribed reference field for a 
target field; a motion vector conversion step of converting a 
motion vector for each of the fields, having a size corresponding 
to a time interval between the target field and the prescribed 
reference field, into a motion vector of a size corresponding to 
a time interval of a fixed unit; an inter-field interpolation pixel 
generation step of obtaining pixels from reference fields which 
are forward and backward fields of a deinterlacing target field 
to be subjected to a deinterlacing process, on the basis of the 
motion vectors converted in the motion vector conversion step, 
and generating a first interpolation pixel for the deinterlacing 
target field; an intra-field interpolation pixel generation step 
of generating a second interpolation pixel using pixels in the 
deinterlacing target field; a weighting factor decision step of 
deciding a weighting factor which indicates a weighting ratio 
between the first interpolation pixel and the second interpolation 
pixel; and a progressive image generation step of obtaining a 
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weighted mean of the first interpolation pixel and the second 
interpolation pixel using the weighting factor, thereby 
generating a third interpolation pixel, and interpolating the 
decoded image using the third interpolation pixel, to generate 
the progressive image. 

According to a 12th aspect of the present invention, there 
is provided a deinterlacing method for performing a decoding 
process to a code sequence, field by field or frame by frame, which 
code sequence is obtained by coding an interlaced image composed 
of plural fields using motion compensation, and converting an 
decoded image of the interlaced image, obtained by the decoding 
process, into a progressive image, comprising: a decoding step 
of decoding the interlaced image, thereby obtaining the decoded 
image as well as obtaining a motion vector at the motion 
compensation which indicates a prescribed reference field for a 
target field; a motion vector conversion step of converting a 
motion vector for each of the fields having a size corresponding 
to a time interval between the target field and the prescribed 
reference field, into a motion vector of a size corresponding to 
a time interval of a fixed unit; a motion vector judgement step 
of judging effectiveness of the motion vectors converted in the 
motion vector conversion step; an inter-field interpolation pixel 
generation step of obtaining pixels from reference fields which 
are forward and backward fields of a deinterlacing target field 
to be subjected to a deinterlacing process, on the basis of the 
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motion vectors converted in the motion vector conversion step and 
a result of the judgement in the motion vector judgement step, 
and generating a first interpolation pixel for the deinterlacing 
target field; an intra-field interpolation pixel generation step 
of generating a second interpolation pixel using pixels in the 
deinterlacing target field; a weighting factor decision step of 
deciding a weighting factor which indicates a weighing ratio 
between the first interpolation pixel and the second interpolation 
pixel; and a progressive image generation step of obtaining a 
weighted mean of the first interpolation pixel and the second 
interpolation pixel using the weighing factor, thereby generating 
a third interpolation pixel, and interpolating the decoded image 
using the third interpolation pixel, to generate the progressive 
image . 

According to a 13th aspect of the present invention, in the 
deinterlacing method of the 11th or 12th aspect, the time interval 
of a fixed unit in the motion vector conversion step is a time 
interval equivalent to one field. 

According to a 14th aspect of the present invention, in the 
deinterlacing method of the 11th or 12th aspect, processes in the 
inter-field interpolation pixel generation step, the weighing 
factor decision step and the progressive image generation step 
are carried out in units, which unit is smaller than a unit of 
an image accompanied by the motion vector at the motion 
compensation . 
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According to a 15th aspect of the present invention , in the 
deinterlacing method of the 11th or 12th aspect, the code sequence 
is a code sequence which is coded by an MPEG method* 

According to a 16th aspect of the present invention, in the 
deinterlacing method of the 11th or 12th aspect, in the motion 
vector conversion step, when a distance between lines in a frame 
structure is one pixel, the motion vector is converted so that 
vertical components of the motion vector have an even number. 

According to a 17th aspect of the present invention, in the 
deinterlacing method of the 12th aspect, in the motion vector 
judgement step, when the size of the motion vector converted in 
the motion vector conversion step is equal to or smaller than a 
predetermined value, the motion vector is judged effective. 

According to an 18th aspect of the present invention, in 
the deinterlacing method of the 12th aspect, in the motion vector 
judgement step, when a distance between lines in a frame structure 
is one pixel, a motion vector which has even-numbered vertical 
components among the motion vectors converted in the motion vector 
conversion step is judged effective. 

According to a 19th aspect of the present invention, in the 
deinterlacing method of the 11th aspect, in the inter-field 
interpolation pixel generation step, an evaluation scale for 
selecting an optimum motion vector for the generation of the first 
interpolation pixel is calculated using the motion vectors 
converted in the motion vector conversion step, and the first 
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interpolation pixel is generated using a motion vector with which 
a best evaluation scale is obtained. 

According to a 2 0th aspect of the present invention, in the 
deinterlacing method of the 11th aspect, in the inter-field 
interpolation pixel generation step, an evaluation scale for 
selecting an optimum motion vector for the generation of the first 
interpolation pixel is calculated using the motion vector 
converted in the motion vector conversion step and a motion vector 
in the opposite direction to the motion vector, and the first 
interpolation pixel is generated using a motion vector with which 
a best evaluation scale is obtained; and the motion vector in the 
opposite direction is a motion vector which is in the opposite 
direction to the motion vector converted in the motion vector 
conversion step and indicates a reference field in an opposite 
forward/backward relationship to the reference field indicated 
by the motion vector with respect to the target field. 

According to a 21st aspect of the present invention, in the 
deinterlacing method of the 12th aspect, in the inter-field 
interpolation pixel generation step, an evaluation scale for 
selecting an optimum motion vector for the generation of the first 
interpolation pixel is calculated using a motion vector which is 
judged effective in the motion vector judgement step, among the 
motion vectors converted in the motion vector conversion step, 
and the first interpolation pixel is generated using a motion 
vector with which a best evaluation scale is obtained. 
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According to a 22nd aspect of the present invention, in the 
deinterlacing method of the 12th aspect, in the inter-field 
interpolation pixel generation step, an evaluation scale for 
selecting an optimum motion vector for the generation of the first 
interpolation pixel is calculated using an effective motion vector 
which is judged effective in the motion vector judgement step and 
a motion vector in the opposite direction to the effective motion 
vector, among the motion vectors converted in the motion vector 
conversion step, and the first interpolation pixel is generated 
using a motion vector with which a best evaluation scale is 
obtained; and the motion vector in the opposite direction is a 
motion vector which is in the opposite direction to the effective 
motion vector and indicates a reference field in an opposite 
forward/backward relationship to the reference field indicated 
by the effective motion vector with respect to the target field. 

According to a 23rd aspect of the present invention, in the 
deinterlacing method of any of the 19th to 2 2nd aspects, in the 
inter-field interpolation pixel generation step, an evaluation 
scale for selecting an optimum motion vector for the generation 
of the first interpolation pixel is calculated using the motion 
vector converted in the motion vector conversion step and a motion 
vector having no motion, and the first interpolation pixel is 
generated using a motion vector with which a best evaluation scale 
is obtained. 

According to a 24th aspect of the present invention, in the 
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deinterlacing method of any of the 19th to 2 2nd aspects, the 
evaluation scale is a sum of absolute values of differences between 
pixels of the reference field which is indicated by the motion 
vector converted in the motion vector conversion step and the 
second interpolation pixels. 

According to a 25th aspect of the present invention, in the 
deinterlacing method of the 23rd aspect, the evaluation scale is 
a sum of absolute values of differences between pixels of the 
reference field which is indicated by the motion vector converted 
in the motion vector conversion step and the second interpolation 
pixels • 

According to a 26th aspect of the present invention, in the 
deinterlacing method of any of the 2 0th to 2 2nd aspects, the 
evaluation scale is a sum of absolute values of differences between 
pixels of the reference field which is indicated by the motion 
vector converted in the motion vector conversion step and pixels 
of a reference field which is indicated by the motion vector in 
the opposite direction. 

According to a 27th aspect of the present invention, in the 
deinterlacing method of any of the 20th to 22nd aspects, in the 
inter-field interpolation pixel generation step, an evaluation 
scale for selection an optimum motion vector for the generation 
of the first interpolation pixel is calculated using the motion 
vector converted in the motion vector conversion step and a motion 
vector having no motion, and the first interpolation pixel is 
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generated using a motion vector with which a best evaluation scale 
is obtained; and the evaluation scale is a sum of absolute values 
of differences between pixels of the reference field which is 
indicated by the motion vector converted in the motion vector 
conversion step and pixels of the reference field which is 
indicated by the motion vector of the opposite direction. 

According to a 28th aspect of the present invention, there 
is provided a deinterlacing method for generating an interpolation 
pixel for an interlaced image which is composed of plural fields, 
using pixels in each of the fields, and converting the interlaced 
image into a progressive image, comprising: an edge detection step 
of detecting a direction indicated by a line passing through a 
position to be interpolated where the interpolation pixel is 
generated and connecting peripheral pixels of the position to be 
interpolated, as a direction of an edge; an edge reliability 
decision step of obtaining a strongness of a correlation between 
pixels existing in the direction of the edge, as a reliability 
of the edge; and an interpolation pixel generation step of 
generating the interpolation pixel using the pixels existing in 
the direction of the edge when the reliability of the edge is equal 
to or larger than a predetermined value, and generating the 
interpolation pixel using pixels existing in upper and lower 
directions of the position to be interpolated when the reliability 
of the edge is smaller than the predetermined value. 

According to a 29th aspect of the present invention, in the 
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deinter lacing method of the 11th or 12th aspect, the intra-field 
interpolation pixel generation step including: an edge detection 
step of detecting a direction indicated by a line passing through 
a position to be interpolated where the second interpolation pixel 
is generated and connecting peripheral pixels of the position to 
be interpolated, as a direction of an edge; an edge reliability 
decision step of obtaining a strongness of a correlation between 
pixels existing in the direction of the edge, as a reliability 
of the edge; and an interpolation pixel generation step of 
generating the second interpolation pixel using the pixels 
existing in the direction of the edge when the reliability of the 
edge is equal to or larger than a predetermined value, and 
generating the second interpolation pixel using pixels existing 
in upper and lower directions of the position to be interpolated 
when the reliability of the edge is smaller than the predetermined 
value . 

According to a 3 0th aspect of the present invention, in the 
deinterlacing method of the 28th aspect, in the edge reliability 
decision step, when a difference between the pixels existing in 
the direction of the edge is smaller than a difference between 
the pixels existing in the upper and lower directions of the 
position to be interpolated, the reliability of the edge is judged 
to be equal to or larger than the predetermined value. 

According to a 31st aspect of the present invention, in the 
deinterlacing method of the 29th aspect, in the edge reliability 



-24- 



decision step, when a difference between the pixels existing in 
the direction of the edge is smaller than a difference between 
the pixels existing in the upper and lower directions of the 
position to be interpolated, the reliability of the edge is judged 
to be equal to or larger than the predetermined value. 

According to a 32nd aspect of the present invention, in the 
deinterlacing method of the 28th aspect, in the edge reliability 
decision step, when the interpolation pixel value which is 
obtained using the pixels existing in the direction of the edge 
is a value between values of the pixels existing in the upper and 
lower directions of the position to be interpolated, the 
reliability of the edge is judged to be equal to or larger than 
the predetermined value. 

According to a 33rd aspect of the present invention, in the 
deinterlacing method of the 29th aspect, in the edge reliability 
decision step, when the interpolation pixel value which is 
obtained using the pixels in the direction of the edge is a value 
between values of the pixels existing in the upper and lower 
directions of the position to be interpolated, the reliability 
of the edge is judged to be equal to or larger than the predetermined 
value. 

According to a 34th aspect of the present invention, in the 
deinterlacing method of the 11th or 12th aspect, for an intra-coded 
deinterlacing target image area in the deinterlacing target field, 
the deinterlacing process is performed using a motion vector which 
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accompanies a peripheral image area positioned around the 
deinter lacing target image area or an image area in a frame 
immediately preceding or immediately following the deinterlacing 
target field, which image area is at the same position as that 
of the deinterlacing target image area. 

According to a 35th aspect of the present invention, in the 
deinterlacing method of the 11th or 12th aspect, when the code 
sequence which is decoded in the decoding step is recorded on a 
recording medium and read in a fast-forward or fast-rewind mode, 
the decoded image is interpolated using only the second 
interpolation pixel generated in the intra-field interpolation 
pixel generation step, thereby generating the progressive image. 

According to a 36th aspect of the present invention, there 
is provided a deinterlacing apparatus for performing a decoding 
process to a code sequence, field by field or frame by frame, which 
code sequence is obtained by coding an interlaced image composed 
of plural fields using motion compensation, and converting a 
decoded image of the interlaced image, obtained by the decoding 
process, into a progressive image, comprising: a decoder for 
decoding the interlaced image, thereby obtaining the decoded image 
as well as obtaining a motion vector at the motion compensation 
which indicates a prescribed reference field for a target field; 
an image memory for storing the decoded image; a parameter memory 
for storing the motion vector; a motion vector converter for 
converting a motion vector for each of the fields, having a size 
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corresponding to a time interval between the target field and the 
prescribed reference field, which is read from the parameter 
memory, into a motion vector of a size corresponding to a time 
interval of a fixed unit; an inter-field interpolation pixel 
generator for obtaining pixels from reference fields which are 
forward and backward fields of a deinterlacing target field to 
be subjected to a deinterlacing process, on the basis of the motion 
vectors converted by the motion vector converter, and generating 
a first interpolation pixel for the deinterlacing target field; 
an intra-field interpolation pixel generator for generating a 
second interpolation pixel using pixels in the deinterlacing 
target field; a weighing factor decision unit for deciding a 
weighting factor which indicates a weighing ratio between the 
first interpolation pixel and the second interpolation pixel; and 
a progressive image generator for obtaining a weighted mean of 
the first interpolation pixel and the second interpolation pixel 
using the weighting factor, thereby generating a third 
interpolation pixel, and interpolating the decoded image read from 
the image memory using the third interpolation pixel, to generate 
the progressive image. 

According to a 37th aspect of the present invention, there 
is provided a deinterlacing apparatus for performing a decoding 
process to a code sequence, field by field or frame by frame, which 
code sequence is obtained by coding an interlaced image composed 
of plural fields using motion compensation, and converting a 



decoded image of the interlaced image, obtained by the decoding 
process, into a progressive image, comprising: a decoder for 
decoding the interlaced image, thereby obtaining the decoded image 
as well as obtaining a motion vector at the motion compensation 
which indicates a prescribed reference field for a target field; 
an image memory for storing the decoded image; a parameter memory 
for storing the motion vector; a motion vector converter for 
converting a motion vector for each of the fields, having a size 
corresponding to a time interval between the target field and the 
prescribed reference field, which is read from the parameter 
memory, into a motion vector of a size corresponding to a time 
interval of a fixed unit; a motion vector judgement unit for judging 
effectiveness of the motion vectors converted by the motion vector 
converter; an inter-field interpolation pixel generator for 
obtaining pixels from reference fields which are forward and 
backward fields of a deinterlacing target field to be subjected 
to a deinterlacing process, on the basis of the motion vectors 
converted by the motion vector converter and a result of the 
judgement by the motion vector judgement unit, and generating a 
first interpolation pixel for the deinterlacing target field; an 
intra-f ield interpolation pixel generator for reading pixels in 
the deinterlacing target field to generate a second interpolation 
pixel; a weighting factor decision unit for deciding a weighting 
factor which indicates a weighting ratio between the first 
interpolation pixel and the second interpolation pixel; and a 
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progressive image generator for obtaining a weighted mean of the 
first interpolation pixel and the second interpolation pixel using 
the weighting factor, thereby generating a third interpolation 
pixel, and interpolating the decoded image read from the image 
memory using the third interpolation pixel, to generate the 
progres s ive image . 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram illustrating a structure of a 
deinterlacing apparatus according to a first embodiment of the 
present invention . 

Figure 2 is a schematic diagram for explaining an interlaced 

image . 

Figure 3(a) to 3(d) are schematic diagrams for explaining 
examples of frequency characteristics of a VT filter 102 according 
to embodiments of the present invention. 

Figure 4 is a block diagram illustrating a structure of a 
deinterlacing apparatus according to a second embodiment of the 
present invention ♦ 

Figure 5 is a block diagram illustrating a structure of a 
deinterlacing apparatus according to a third embodiment of the 
present invention . 

Figure 6 is a block diagram illustrating a deinterlacer 508 
according to embodiments of the present invention. 

Figure 7 is a schematic diagram illustrating a state of an 
interlaced image. 
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Figure 8 is a flowchart for explaining a judgement method 
of a motion vector judgement unit 6 03 according to embodiments 
of the present invention. 

Figure 9 is a flowchart for explaining an example of an 
operation of an inter-field interpolation pixel generator 604 
according to embodiments of the present invention. 

Figure 10 is a flowchart for explaining an example of an 
operation of an inter-field interpolation pixel generator 604 
according to embodiments of the present invention. 

Figure 11 is a flowchart for explaining an example an 
operation of an inter-field interpolation pixel generator 6 04 
according to embodiments of the present invention. 

Figure 12 is a flowchart for explaining an example of an 
operation of an inter-field interpolation pixel generator 604 
according to embodiments of the present invention. 

Figure 13 is a diagram schematically showing a state of an 
interlaced image. 

Figure 14 is a diagram schematically showing a state of an 
interlaced image. 

Figure 15 is a diagram schematically showing a state of an 
interlaced image. 

Figure 16 is a diagram schematically showing a state of an 
interlaced image. 

Figure 17 is a diagram schematically showing a state of an 
interlaced image. 
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Figure 18 is a diagram schematically showing a state of an 
interlaced image. 

Figure 19 is a diagram schematically showing a state of a 
deinterlacing target field pixel. 

Figure 2 0 is a diagram schematically showing a state of a 
deinterlacing target field pixel. 

Figure 21 is a flowchart for explaining an example of an 
operation of an intra-field interpolation pixel generator 605 
according to embodiments of the present invention. 

Figure 22 is a flowchart for showing an operation of a 
weighting factor decision unit 606 according to embodiments of 
the present invention. 

Figure 23 is a diagram schematically showing a deinterlacing 
target frame including an intra-coded macroblock, and forward and 
backward frames. 

Figure 24 is a block diagram illustrating a structure of 
a deinterlacer 50 8 according to a fifth embodiment of the present 
invention. 

Figure 25 is a diagram schematically illustrating an 
interlaced image for explaining a prior art deinterlacing method. 
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Hereinafter, embodiments of the present invention will be 
described with reference to the drawings. 
[Embodiment 1] 

Hereinafter, a first embodiment of the present invention 



will be described with reference to figures 1 and 2. Figure 1 is 
a block diagram illustrating a deinterlacing apparatus that 
includes a frame memory 101, a VT filter unit 102 , a filter 
coefficient setting unit 103 , a difference operation unit 104 and 
a double-speed converter 105. Figure 2 is a diagram schematically 
showing a state of an interlaced image on a time-vertical plane, 
in which a white dot shows a pixel (scan line) and white dots which 
are arranged in the vertical direction show pixels in the same 
field. 

Initially, the deinterlacing apparatus stores an inputted 
interlaced image in the frame memory 101. Here, the description 
is given of a case where fields #n-2 to #n+l shown in figure 2, 
i.e., four fields are stored, and the field #n is subjected to 
deinterlacing . 

In this case, the VT filter unit 102 receives data of the 
fields #n-l, #n and #n+l. Then, when an interpolation pixel in 
a position "k" in figure 2 is to be generated, pixels adjacent 
to the position "k M are subjected to the filtering process. Here, 
pixels ,, a ,, ~ n j ,, are subjected to the filtering process. 

In addition, the difference operation unit 104 receives data 
of two frames including the field #n (deinterlacing target) , i.e. , 
frames #m-l and #m ( fields #n-2 to #n+l ) . The difference operation 
unit 104 calculates a difference between the frames #m-l and #m 
to obtain the quantity of motion of the frame #n. Difference 
values between positionally corresponding pixels in these frames 
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are obtained, and the sum of respective absolute values of the 
difference values is obtained as the quantity of motion. Then, 
the obtained result is outputted to the filter coefficient setting 
unit 103. 

The filter coefficient setting unit 103 decides a filter 
coefficient on the basis of the sum of absolute values of the 
differences inputted from the difference operation unit 104 . For 
example, thresholds TH1, TH2 and TH3 (assuming that TH1<TH2<TH3) 
are set, and when the sum of absolute values of the differences 
is smaller than the threshold TH1, the interlaced image is judged 
to be a still picture. When the sum is equal to or larger than 
THl and smaller than TH2, the image is judged to be a quasi-still 
picture. When the sum is equal to or larger than TH2 and smaller 
than TH3, it is judged to be a quasi-moving picture. When the sum 
is equal to or larger than TH3, it is judged to be a moving picture. 
Then, the predetermined filter coefficient corresponding to each 
of a still picture, quasi-still picture, quasi-moving picture and 
moving picture is outputted to the VT filter unit 102. 

Here, the filter coefficient setting unit 103 fundamentally 
sets the coefficient so that the VT filter unit 102 extracts 
vertical low frequency components from the deinterlacing target 
field and extracts vertical high frequency components from its 
adjacent field. For example, when an interpolation pixel "k" as 
shown in figure 2 is to be generated, the filter coefficient setting 
unit 102 sets the coefficient so that the VT filter unit 102 



-33- 



extracts vertical high frequency components from the pixels 
"a"-"c" and "h"-" j M and extracts vertical low frequency components 
from the pixels "d n ~"g ,, . 

Further, the filter coefficient setting unit 103 sets the 
filter coefficient so that components (gain) from the adjacent 
fields are decreased in the order of still picture, quasi-still 
picture, quasi-moving picture, and moving picture. For example, 
in the case of moving picture, the filter coefficient may be set 
so that the components (gain) from the adjacent fields become zero. 
Examples of the frequency characteristics of the VT filter unit 
102 in this case are shown in figures 3(a) to 3(d). Figures 3 are 
diagrams showing the gain with respect to the vertical frequency 
in the filter. Figure 3(a) shows the case of a still picture, 
figure 3(b) shows the case of a quasi-still picture, figure 3(c) 
shows the case of a quasi-moving picture, and figure 3(d) shows 
the case of a moving picture. In figures 3, the full line shows 
the filter characteristic to a deinterlacing target field and the 
broken line shows the frequency characteristics to the adjacent 
field. 

The VT filter unit 102 filters the inputted pixels using 
the filter coefficient value which is inputted from the filter 
coefficient setting unit 103, to generate an interpolation pixel, 
and outputs the generated interpolation pixel. The generated 
interpolation pixel is inputted to the double-speed converter 105. 
To the double-speed converted 105, the pixel data of the 
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deinterlacing target field is inputted from the frame memory and 
the interpolation pixel data is inputted from the VT filter unit 
102 * The double-speed converter 105 composes these data, converts 
the frame rate to be doubled , and outputs the converted data as 
a progressive image. 

As described above , in this first embodiment, when an 
interlaced image is to be converted into a progressive image, the 
VT filter unit 102 subjects pixels of a deinterlacing target field 
and forward and backward fields to the VT filtering, thereby 
generating the interpolation pixel. At that time, the difference 
operation unit 104 obtains the sum of the absolute values of 
differences between pixels of a frame including the deinterlacing 
target field and its immediately preceding frame, and the filter 
coefficient setting unit 103 decides the coefficient of the VT 
filter on the basis of the sum of the absolute values. The 
coefficient is decided so that the image is judged to be a moving 
picture when the sum of the absolute values of the differences 
is larger, and then the contribution (gain) from the adjacent 
fields is reduced. 

Therefore, in the case of still picture, a progressive image 
of high resolution can be obtained like in the case of using the 
prior art VT filtering method, and in the case of moving picture, 
the image quality of a part where the image quality is degraded 
by the prior art VT filter can be greatly improved. Further, these 
operations can be implemented by one filter, whereby the costs 
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can be reduced. 

Further, in this first embodiment, the filter coefficient 
setting unit 103 judges which one of a still picture, quasi-still 
picture, quasi-moving picture and moving picture the interlaced 
image is, on the basis of the absolute values of the differences 
inputted from the difference operation unit 104, and sets the 
coefficient at four levels. Therefore, as compared to the prior 
art deinterlacing method by which the filtering coefficient are 
'tf switched between two levels according to whether the interlaced 
ft image is a still picture or a moving picture, an interpolation 
2 pixel that matches the motion of the image more can be generated. 
T In this first embodiment, when the quantity of motion of 

S a deinterlacing target field is obtained, the difference operation 
lf% unit 104 obtains the sum of the absolute values of the differences 
between a frame including the deinterlacing target field and the 
forward frame, while the unit 104 may obtain the sum of absolute 
values of differences between the deinterlacing target field and 
the forward field. Further, the unit 104 may obtain the sum of 
absolute values of differences between the frame including the 
deinterlacing target field and a backward frame, or between the 
deinterlacing target field and a backward field. Further, the 
unit 104 may obtain a larger one of the sums of the absolute values 
of the differences between the frame including the deinterlacing 
target field and the forward and backward frames or the sums of 
the absolute values of the differences between the deinterlacing 
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target field and the forward and backward fields , or the mean value 
of these sums. 

In addition, in this first embodiment, the VT filter unit 
102 performs the filtering using 10 pixels to generate an 
interpolation pixel, while the number of pixels can be other 
values. 

Further in this first embodiment, the VT filter unit 102 
generates an interpolation pixel of the field #n using the pixels 
of three fields, i.e., the fields #n-l, #n and #n+l, while the 
interpolation pixel may be generated using pixels of two fields, 
such as fields #n-l and #n, or fields #n and #n+l . Since the frame 
memory 101 contains the interlaced image in field units, the size 
of the frame memory in this case can be reduced more as compared 
to the case where the interpolation pixel is generated using the 
pixels of three fields, and the processing quantity at the 
filtering process can be reduced. 

In this first embodiment, the filter coefficient setting 
unit 103 decides the quantity of motion at four levels, while the 
number of levels can be other values* 

Furthermore, in this first embodiment, the quantity of 
motion is decided in field units or frame units, while the filter 
coefficient may be decided by dividing the screen into several 
areas and detecting the quantity of motion for each divided area* 
[Embodiment 2] 

Hereinafter, a second embodiment of the present invention 
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will be described with reference to figure 4. Figure 4 is a block 
diagram illustrating a deinterlacing apparatus including a frame 
memory 101, a VT filter unit 102, a filter coefficient setting 
unit 103, a difference operation unit 404, and a double-speed 
converter 105. 

Initially, the deinterlacing apparatus stores an inputted 
interlaced image into the frame memory 101 . Here, the description 
is given of a case where three fields, i.e., fields #n-l to #n+l 
shown in figure 2 are stored, and the field #n is subjected to 
deinterlacing . 

In this case, the VT filter unit 102 receives data of the 
fields #n-l, #n and #n+l. Then, when an interpolation pixel in 
a position "k" in figure 2 is to be generated, for example pixels 
"a r *~" j" as its adjacent pixels are subjected to the filtering 
process . 

The difference operation unit 404 receives pixels "b" and 
"i" in its adjacent fields, which are at the same position as the 
position to be interpolated, among the pixels to be used by the 
VT filter unit 102 for the filtering process. The difference 
operation unit 404 calculates the absolute value of the difference 
between these pixel values. Then, the obtained result is 
outputted to the filter coefficient setting unit 103. 

The filter coefficient setting unit 103 decides a filter 
coefficient on the basis of the absolute value of the difference 
inputted from the difference operation unit 404. For example, 
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thresholds THl , TH2 and TH3 (assuming that THl<TH2<TH3) are set 
and when the absolute value of the difference is smaller than the 
threshold THl, the interlaced image is judged to be a still picture- 
When the absolute value is equal to or larger than THl and smaller 
than TH2 , the image is judged to be a quasi-still picture. When 
the absolute value is equal to or larger than TH2 and smaller than 
TH3, it is judged to be a quasi-moving picture. When the absolute 
value is equal to or larger than TH3 , it is judged to be a moving 
picture. Then, the predetermined filter coefficient 
corresponding to each of a still picture, quasi-still picture, 
quasi-moving picture and moving picture are outputted to the VT 
filter unit 102. As described above, in this second embodiment, 
the difference operation unit 404 obtains the absolute value of 
the difference between pixels in adjacent fields, which pixels 
are at the same position as the position to be interpolated. 
Therefore, the filter coefficient setting unit 103 can judge the 
quantity of motion for each pixel. Accordingly, when there is a 
moving object or the like in the screen, the image quality 
degradation which has been caused by the prior art VT filter can 
be prevented for that object. Here, the characteristics of the 
coefficient of the VT filter unit 102 are the same as those in 
the first embodiment. 

The VT filter unit 102 subjects the inputted pixels to the 
filtering using the filter coefficient values inputted from the 
filter coefficient setting unit 103, to generate an interpolation 
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pixel, and outputs the generated interpolation pixel* The 
generated interpolation pixel is inputted to the double-speed 
converter 105. To the double-speed converter 105, pixel data of 
the deinterlacing field is inputted from the frame memory 101 and 
the interpolation pixel data is inputted from the VT filter unit 
102. The double-speed converter 105 composes these data, converts 
the frame rate to be doubled, and outputs the converted data as 
a progressive image. 

As described above, in this second embodiment, when an 
interlaced image is to be converted into a progressive image, the 
VT filter unit 102 filters pixels of a deinterlacing target field 
and forward and backward fields, thereby generating the 
interpolation pixel. At that time, the difference operation unit 
404 calculates the absolute value of the difference between the 
pixels of its adjacent fields, which pixels are the same position 
as a position to be interpolated, and the filter coefficient 
setting unit 103 decides the coefficient of the VT filter on the 
basis of the calculated absolute value. The coefficient is 
decided so that the image is judged to be a moving picture when 
the absolute value of the difference is larger, and then the 
contribution from the adjacent fields is reduced. 

Therefore, in the case of still picture, a progressive image 
of high resolution can be obtained like in the case of using the 
prior art VT filter, as well as when there is a moving object or 
the like in the screen, the image quality degradation which is 
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caused by the prior art VT filter can be prevented for that object. 
Further, these operations can be implemented by one filter, 
whereby the costs can be reduced. 

In this second embodiment, the difference operation unit 
4 04 calculates the difference between pixels "b" and "i", while 
a difference between other pixels may be calculated. For example, 
the sum of the absolute values of differences between positionally 
corresponding pixels of the pixels "a n ~"c M and pixels M h"~" j" may 
be calculated, or the absolute value of a difference between the 
mean value of the pixels "e" and "f" and the pixel "b" may be 
calculated . 

Further, in this second embodiment, the VT filter unit 102 
performs the filtering using 10 pixels, thereby generating the 
interpolation pixel, while the number of pixels can be other 
values . 

In this second embodiment, the VT filter unit 102 generates 
the interpolation pixel of the field #n using the pixels of the 
three fields, i.e. fields #n-l, #n and #n+l, while the 
interpolation pixel can be generated using pixels of two fields 
such as fields #n-l and #n, or fields #n and #n+l. In this case, 
since the frame memory 101 contains the interlaced image in field 
units, the size of the frame memory can be reduced more and the 
processing quantity at the filtering process can be reduced more, 
as compared to the case where the interpolation pixel is generated 
using pixels of three fields. 



Furthermore , in this second embodiment, the filter 
coefficient setting unit 103 decides the quantity of motion at 
four levels, while the number of levels can be other values. 
[Embodiment 3] 

Hereinafter, a third embodiment of the present invention 
will be described* Figure 5 is a block diagram illustrating a 
structure of a deinterlacing apparatus including an MPEG video 
decoder 501 and a deinterlacer 508, MPEG video which is obtained 
by motion-compensating and predictive coding an interlaced image 
according to the MPEG method is inputted to this deinterlacing 
apparatus* The MPEG video decoder 501 includes a variable-length 
decoder 502, an inverse quantizer 503, an inverse DCT unit 504, 
an adder 505, a motion compensator 506, and an image memory 5 07 . 
Figure 6 is a block diagram illustrating a structure of the 
deinterlacer 508, which includes a parameter memory 601, a motion 
vector converter 602, a motion vector judgement unit 603, an 
inter-field interpolation pixel generator 6 04, an intra-field 
interpolation pixel generator 605, a weighting factor decision 
unit 606, and a progressive image generator 607. 

An MPEG video code sequence inputted to the MPEG video 
decoder 501 is subjected to variable-length decoding by the 
variable-length decoder 502. DCT coefficients and the like which 
are obtained by variable-length-decoding the MPEG video code 
sequence are outputted to the inverse quantizer 503 . Further, the 
variable-length decoder 502 variable-length-decodes the MPEG 
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video, thereby obtaining information such as the motion vector, 
picture type, macroblock type, temporal reference and motion 
vertical field selection (MVFS), and outputs the obtained 
information to the motion compensator 506 and the deinterlacer 
508. Here, the picture type information indicates which one of 
I picture, P picture and B picture the decoded image data is. The 
macroblock type information indicates by which prediction mode 
(no prediction, forward prediction, backward prediction, 
bidirectional prediction or the like) for example a macroblock 
having the size of 16 x 16 pixels among the decoded image data 
is coded. The temporal reference information indicates the frame 
number of a picture. Further, the MVFS information indicates 
whether a reference image in field prediction is a top or bottom 
field image in a case where an interlaced image is coded by the 
frame structure and the field prediction is used as the motion 
compensation prediction. 

The inverse quantizer 503 subjects the inputted DCT 
coefficients to the inverse quantization. Then, the inverse DCT 
unit 504 subjects the obtained result to the inverse DCT process, 
to generate image data, and outputs the generated image data to 
the adder 505. 

The motion compensator 506 reads reference image data from 
the image memory 507 and motion-compensates the reference image 
data. As will be described later, decoded image data outputted 
from the adder 505 is stored in the image memory 507 . This decoded 
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image data is used as the reference image data at the motion 
compensation* Further, for example according to the MPEG2 coding 
method, the image data is motion-compensated in units of 
macroblock (16x16 pixels ) . Thus , when the inputted code sequence 
is a code sequence which is coded by the MPEG2 coding method, the 
reference image data which is motion-compensated by the motion 
compensator 506 indicates a decoded macroblock (16x16 pixel data) . 
When the decoded macroblock is an intra ( intra-f rame coded) 
macroblock, the motion compensator 506 does not operate. Further, 
when the decoded macroblock is a non-intra macroblock, the motion 
compensator 506 performs the motion compensation on the basis of 
the motion vector inputted from the variable-length decoder 5 02, 
and outputs motion-compensated image data to the adder 50 5. 

When the decoded macroblock is an intra macroblock, the adder 
505 does not process the image data inputted from the inverse DCT 
unit 504 and outputs the data as it is. On the other hand, when 
the decoded macroblock is a non-intra macroblock, the adder 505 
adds the image data inputted from the inverse DCT unit 504 and 
the reference image data from the motion compensator 506, and 
outputs the obtained result. The image data outputted from the 
adder 5 05 is stored in the image memory 5 07 as the decoded image 
data. 

The deinterlacer 508 reads the decoded image data from the 
image memory 507 successively in the order of time, and processes 
the read image data. This operation will be described 
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hereinaf ter . 

To the deinterlacer 508, the information such as the motion 
vector, picture type, temporal reference, macroblock type, and 
MVFS is inputted from the variable-length decoder 502, and the 
decoded image data is inputted from the image memory 507. The 
information, such as the motion vector, picture type, macroblock 
type, temporal reference and MVFS, inputted from the 
variable-length decoder 5 02 is inputted to the parameter memory 
601 and temporarily stored. This is for the purpose of 
compensating the time difference, because the MPEG video decoder 
501 processes each frame in the order of code sequence while the 
deinterlacer 508 processes the image data in the order of time. 

The motion vector converter 602 reads the information such 
as the motion vector, picture type, temporal reference, and MVFS 
from the parameter memory 601, and converts the size of the motion 
vector. Here, the motion vector converter 6 02 can obtain 
information as to how many fields the deinterlacing target field 
which is subjected to the deinterlacing and a reference field which 
has been referred to by the deinterlacing target field at the motion 
compensation in the coding are apart from each other, on the basis 
of the information of the picture type, temporal reference, and 
MVFS. 

Hereinafter, the operation for converting the motion vector 
will be described with reference to figure 7. 

Figure 7 is a diagram schematically showing a state of a 



decoded image (interlaced image) . White dots show arrangement of 
scan lines (pixels) seen from the horizontal direction of the 
screen, and the ordinate axis represents the vertical direction 
and the abscissa axis represents the time direction. That is, 
white dots arranged vertically in a line show scan lines (pixels) 
in the same field. Here, the description is given of a case where 
a motion vector of a macroblock "a" in a deinterlacing target field 
is converted. Assuming that the motion vector of the macroblock 
H a" is a motion vector "A", it means that the macroblock "a" has 
referred to a reference area "b" in a reference field which is 
three frames ( six fields ) apart from the target field at the coding • 
The motion vector converter 602 converts this motion vector "A" 
into a motion vector in one field unit. That is, the motion vector 
f, A" is converted into a motion vector "B" shown by a broken line. 
Then, the converted motion vector is outputted to the motion vector 
judgement unit 603. Only the vertical motion is shown in figure 
7 , while the horizontal motion is similarly processed. Further, 
only the forward motion vector is shown in figure 7, while when 
the macroblock has a backward motion vector the motion vector 
converter 602 carries out the same conversion process to the 
backward motion vector. 

The motion vector judgement unit 603 judges the 
effectiveness of the motion vector outputted by the motion vector 
converter 602. The flowchart for explaining the judgement 
procedure is shown in figure 8. Initially, in step S801, it is 



judged whether the size of the motion vector is equal to or smaller 
than a threshold. The judgement can be "Yes" when both of 
horizontal and vertical motion vectors are equal to or smaller 
than thresholds, or can be "Yes" when the sum of squares of the 
horizontal and vertical motion vectors is equal to or smaller than 
a threshold. When the judgement result in step S801 is "Yes", it 
is judged whether the vertical motion vector value is an even number 
(step S802). Here, as the unit of the motion in the vertical 
direction, the distance between lines in the frame structure is 
assumed one pixel. For example, the vertical size of the motion 
vector "B" in figure 7 is two pixels. The reason why a motion vector 
having an even vertical motion vector value is selected in step 
S802 is because when the vertical motion vector value is an odd 
number, an interpolation pixel cannot be taken directly from the 
reference field. When the judgement result in step S801 is "No", 
the motion vector is judged invalid in step S804 and is not used 
in the following processes . When the judgement result in step S802 
is "Yes" , the motion vector that satisfies the conditions in steps 
S801 and S802 is judged an effective motion vector in step S803 
and used in the following processes. Further, when the judgement 
result in step S802 is "No", the motion vector is judged invalid 
in step S804 and is not used in the following processes. Here, 
when the macroblock has plural motion vectors (forward and 
backward), the judgement can be made individually for the both 
vectors, or can be made simultaneously. The result of the 
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judgement by the motion vector judgement unit 603 and the motion 
vector are outputted to the inter-field interpolation pixel 
generator 604 . 

The inter-field interpolation pixel generator 604 obtains 
a pixel from a reference field on the basis of the motion vector 
and its judgement result inputted from the motion vector judgement 
unit 603, to generate an inter-field interpolation pixel for the 
deinterlacing target field- The subsequent processes are 
performed in units of macroblock, or units which are smaller than 
the macroblock. In the following description, assume that the 
processes are performed in units, each unit being composed of 8 
pixels (horizontal) x 4 pixels (vertical) (hereinafter, referred 
to as a block). Further, when simply referred to as "a motion 
vector" below, it indicates a motion vector which has been already 
converted by the motion vector converter 602. 

Examples of the operation of the inter-field interpolation 
pixel generator 604 are shown in figures 9 and 10. When there are 
plural effective motion vectors, the inter-field interpolation 
pixel generator 604 initially calculates an evaluation scale for 
selecting one of the motion vectors, which is to be used for 
generating an inter-field interpolation pixel. In this third 
embodiment, two calculation methods for obtaining the evaluation 
scale will be described. Figure 9 is a flowchart for explaining 
an example of the operation of the inter-field interpolation pixel 
generator 604 when one of the motion vectors is selected on a first 
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evaluation scale. Figure 10 is a flowchart for explaining an 
example of the operation of the inter-field interpolation pixel 
generator 604 when one of the motion vectors is selected on a second 
evaluation scale. 

Hereinafter, the case where the motion vector is selected 
on the first evaluation scale is described with reference to figure 
9. 

Initially in step S901, it is judged whether or not the 
deinterlacing target block includes an effective forward motion 
vector. Here, the case where an effective forward motion vectors 
is included is a case where a macroblock including the 
deinterlacing target block has a forward motion vector and the 
motion vector judgement unit 603 judges the forward motion vector 
to be effective. When the judgement result in step S901 is "Yes", 
the operation proceeds to step S902. When the result is "No", the 
operation proceeds to step S903. In step S902, the evaluation 
scale is calculated using the forward motion vector. 

Hereinafter, the first evaluation scale calculation method 
using the forward motion vector will be described with reference 
to figure 13. 

Figure 13 is a diagram schematically showing a state of a 
decoded image (interlaced image) as in figure 7. Assume here that 
an inter-field interpolation pixel is generated for a block "c" 
in the deinterlacing target field, and the forward motion vector 
of a macroblock including the block "c" is a motion vector "C". 
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In this case, the sum of absolute values of differences between 
a block "d" in a forward reference field indicated by the motion 
vector "C" and a block "e" in a backward reference field indicated 
by a motion vector "C" that is in the opposite direction to the 
motion vector "C" is calculated. Here, the opposite motion vector 
"C" is a motion vector which is in the opposite direction to the 
motion vector "C" and indicates a reference field having an 
opposite positional (forward or backward) relationship to the 
reference field indicated by the motion vector "C", with respect 
to the deinterlacing target field. Further, the calculation of 
the sum of the absolute values of differences between blocks means 
that differences between corresponding pixels which are coupled 
by arrows in the blocks "d" and "e" are obtained and then the sum 
of the absolute values of these differences is obtained in block 
unit. Then, the obtained sum of the absolute values of the 
differences is used as the first evaluation scale. In figure 13, 
only vertical pixels are shown, while horizontal pixels are also 
targets for the calculation of the sum of the absolute values of 
differences between pixels as the first evaluation scale. 

In step S903, it is judged whether or not the block "c" 
includes an effective backward motion vector. When the judgement 
result in step S903 is "Yes" the operation proceeds to step S904, 
and when the result is "No" the operation proceeds to step S905. 

In step S904, the evaluation scale is calculated using the 
backward motion vector. Since this evaluation scale calculation 
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method is the same as that in step S902, its description is not 
given here. It is different in that the backward motion vector 
is used. 

Finally in step S905, on the basis of the evaluation scales 
obtained in the above-mentioned processes, the interpolation 
method is selected, i.e. , a motion vector to be used for generating 
the inter-field interpolation pixel is selected, and then the 
inter-field interpolation pixel is generated. The selection 
method will be described later. 

Next, the case where the motion vector is selected on the 
second evaluation scale will be described with reference to figure 
10. 

Initially in step S1001, it is judged whether or not a 
deinterlacing target block includes an effective forward motion 
vector. When the judgement result in step S1001 is "Yes" the 
operation proceeds to step S1002, and when the result is "No" the 
operation proceeds to step S1006. In step S1002, the evaluation 
scale is calculated using the forward motion vector. 

Hereinafter, the second evaluation scale calculation method 
will be described with reference to figure 14. Figure 14 is a 
diagram schematically showing a state of a decoded image 
(interlaced image) as in figure 7. Assume here that an inter- 
field interpolation pixel is generated for a block "c" and a 
macroblock including the block "c" in the deinterlacing target 
field has only a forward motion vector "C" as an effective motion 
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vector. In this case, the sum of the absolute values of 
differences between pixels in the block "d" in a forward reference 
field indicated by the motion vector "C" and interpolation pixels 
in the block "c", generated by the intra-f ield interpolation pixel 
generator 605 (the generation method will be described later), 
is calculated. Here, the calculation of the sum of the absolute 
values of differences between blocks means that differences 
between corresponding pixels which are coupled by arrows in the 
blocks "d" and "c" are obtained and then the sum of the absolute 
values of these differences is obtained in block unit. Then, the 
sum of the absolute values of these differences is used as the 
second evaluation scale. In figure 14, only vertical pixels are 
shown, while horizontal pixels are also targets for the 
calculation of the sum of the absolute values of differences 
between pixels as the second evaluation scale. 

Another method for calculating the second evaluation scale 
will be described with reference to figure 15. Assume here that 
an inter-field interpolation pixel is generated for a block "c" 
and a macroblock including the block "c" has only forward motion 
vector "C M as an effective motion vector. In this case, the sum 
of the absolute values of differences between pixels in a block 
n e ff in a backward reference field indicated by a motion vector 
"C" that is in the opposite direction to the motion vector n C" 
and intra-f ield interpolation pixels in the block "c", generated 
by the intra-f ield interpolation pixel generator 605 (which will 



1 1 iiiiii 



-52- 



be described later), is calculated. Here, the calculation of the 
sum of the absolute values of the differences between blocks means 
that differences between corresponding pixels which are coupled 
by arrows in the blocks "c" and "e" are obtained and then the sum 
of the absolute values of these differences is obtained in block 
unit. Then, the sum of the absolute values of the differences is 
used as the second evaluation scale. In figure 15, only vertical 
pixels are shown, while horizontal pixels are also targets for 
the calculation of the sum of the absolute values of differences 
between the pixels as the second evaluation scale. 

Further, as another method for calculating the second 
evaluation scale, there can be considered a method by which in 
figure 15 the sum of the absolute values of differences between 
pixels generated from mean values of positionally corresponding 
pixels in the blocks "d" and "e" and intra-field interpolation 
pixels in the block "c", generated by the intra-field 
interpolation pixel generator 605 (which will be described later) , 
is obtained and the sum of the absolute values of the differences 
is used as the second evaluation scale. 

Subsequently in step S1003, after the evaluation scale of 
the effective forward motion vector is calculated, it is further 
judged whether or not the deinterlacing target block includes an 
effective backward motion vector. When the judgement result in 
step S1003 is "Yes", the operation proceeds to step S1004 . When 
the result is "No", the operation proceeds to step S1008. 
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In step SI 004, the evaluation scale is calculated using the 
backward motion vector. Since the evaluation scale calculation 
method is the same as that in step S1002, its description is not 
given here. It is difference in that the backward motion vector 
is used. 

In step S1005, the evaluation scale is calculated using 
motion vectors of two directions. This evaluation scale 
calculation method will be described with reference to figure 16. 
Assume here that an inter-field interpolation pixel is generated 
for a block "c" and a macroblock including the block "c" has a 
forward motion vector "C M and a backward motion vector "D" as the 
motion vectors. In this case, initially, mean values of pixels 
in a block "d" in a forward reference field indicated by the motion 
vector "C" and pixels in a block "e" in a backward reference field 
indicated by the motion vector "D" are respectively calculated, 
and thereafter the sum of the absolute values of differences 
between the mean values and interpolation pixels in the block "c" , 
generated by the intra-field interpolation pixel generator 605 
(which will be described later), is calculated. Here, the 
calculation of the sum of the absolute values of differences 
between blocks means that differences between the mean values of 
corresponding pixels which are connected by arrows in the blocks 
"d" and "e" and the intra-field interpolation pixels which are 
connected by the arrows in the block "c" are obtained, and then 
the sum of the absolute values of these differences is obtained 
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in block unit. Then, the sum of the absolute values of the 
differences is used as the second evaluation scale. In figure 16, 
only vertical pixels are shown, while horizontal pixels are also 
targets for the calculation of the sum of absolute values of 
differences between pixels as the second evaluation scale. 

When these is no effective forward motion vector, it is 
judged whether or not the deinterlacing target block includes an 
effective backward motion vector (step SI 006). When the judgement 
result in step SI 006 is "Yes" , the operation proceeds to step S1007 . 
When the result is "No", the operation proceeds to step S1008. 

In step S1007, the evaluation scale is calculated using the 
backward motion vector. Since the evaluation scale calculation 
method is the same as that in step S1004, the description is not 
given here. 

Finally in step S1008, on the basis of the evaluation scales 
obtained in the above-mentioned processes, the interpolation 
method is selected, i.e., a motion vector which is to be used and 
a reference field from which the inter-field interpolation pixel 
is to be obtained are selected, and then the inter-field 
interpolation pixel is generated. The selection method will be 
described later. 

The operation for generating the inter-field interpolation 
pixel on the basis of the obtained evaluation scales will be 
described hereinafter . 

Initially, the processing method in step S905 in the case 
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where the first evaluation scale is used is described. In this 
processing method, when there are forward and backward motion 
vectors as effective motion vectors, a motion vector from which 
the minimum first evaluation scale is obtained is selected. Then, 
an inter-field interpolation pixel is generated using the motion 
vector from which the minimum evaluation scale is obtained. As 
an example, an inter-field interpolation pixel generation method 
in a case where the evaluation scale of the forward motion vector 
is the minimum will be described with reference to figure 17. In 
this case, mean values of a block "d" referred to by a forward 
motion vector "C" from a forward reference field, and a block M e" 
referred to by a motion vector "C" that is in the opposite 
direction to the forward motion vector "C" from a backward 
reference field are generated as the inter-field interpolation 
pixels. Here, the calculation of the mean values between the 
blocks indicates that mean values of pixels which are connected 
by arrows in the blocks "d" and "e" in figure 17 are obtained. 
In figure 17, only vertical pixels are shown, while also as for 
horizontal pixels, the contents of the processing are the same. 
Further, in this case, the pixels in the block "d" or the pixels 
in the block "e" may be used as they are, as the inter-field 
interpolation pixels. 

Next, the processing method in step S1008 in the case where 
the second evaluation scale is used will be described. According 
to this processing method, when there is a forward motion vector, 
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a backward motion vector, or motion vectors in the two directions 
as the effective motion vector , a motion vector and an 
interpolation direction with which the minimum of the second 
evaluation scales obtained for one direction, the opposite 
direction, or two directions with respect to each motion vector 
is obtained are selected. Then, the inter-field interpolation 
pixels are generated on the basis of the selected motion vector 
and interpolation direction. Here, the selection of the 
interpolation direction means that a reference field in a 
direction indicated by the motion vector, from which field the 
interpolation pixels are to be obtained, is selected. Here, one 
of directions (one direction, the opposite direction, and two 
directions) in which the evaluation scale is obtained using the 
corresponding motion vector is selected as the interpolation 
direction. As an example, an inter-field interpolation pixel 
generation method in a case where an evaluation scale which is 
obtained for two directions using a forward motion vector is the 
minimum will be described with reference to figure 17. Figure 17 
shows the case where the evaluation scale that is obtained for 
two directions with using the forward motion vector "C" is the 
minimum. In this case, the mean values of the block "d" referred 
to by the forward motion vector "C" and the block "e M referred 
to by the motion vector "C" that is in the opposite direction 
to the forward motion vector "C" from a backward reference field 
are generated as the inter-field interpolation pixels. Here, the 
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calculation of the mean values between blocks means that mean 
values of corresponding pixels which are coupled by arrows in the 
blocks "d" and "e" are obtained. When the evaluation scale that 
is obtained using the forward motion vector "C" with respect to 
one direction is the minimum, the inter-field interpolation pixels 
are generated from pixels in the block "d" , and when the evaluation 
scale that is obtained using the forward motion vector "C" with 
respect to the opposite direction is the minimum, the inter-field 
interpolation pixels are generated from pixels in the block "e". 
In figure 17, only vertical pixels are shown, while also as for 
horizontal pixels, the same processing is performed. 

The inter-field interpolation pixels may be generated using 
both of the first and second evaluation scales. In this case, for 
example a method by which the motion vector is selected on the 
first evaluation scale and the interpolation direction is decided 
on the second evaluation scale can be employed. 

Next, examples of the operation of the inter-field 
interpolation pixel generator 6 04 in a case where the evaluation 
scale for selecting the optimum motion vector for the generation 
of inter-field interpolation pixels is calculated using a motion 
vector converted by the motion vector converter 602 and a motion 
vector having no motion (static motion vector), and the 
inter-field interpolation pixels are generated using the motion 
vector with which the best evaluation scale is obtained. Figure 
11 is a flowchart for explaining an example of the operation of 
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the inter-field interpolation pixel generator 604 when the motion 
vector is selected on the first evaluation scale* Figure 12 is 
a flowchart for explaining an example of the operation of the 
inter-field interpolation pixel generator 604 when the motion 
vector is selected on the second evaluation scale. 

Initially, an example of the operation of the inter-field 
interpolation pixel generator 604 is described with reference to 
figure 11. 

In figure 11, since the operations in steps S901 to S904 
are the same as those in figure 9, the descriptions are not given 
here. In step S1101, it is judged whether or not an effective 
motion vectors (forward or backward) includes a static motion 
vector. When the judgement result in step S1101 is "Yes", the 
operation proceeds to step S1102. When the result is "No", the 
operation proceeds to step S1103. 

In step SI 102 , the first evaluation scale is calculated using 
the static motion vector. The calculation method is described 
with reference to figure 18. Figure 18 is a diagram schematically 
showing a state of a decoded image (interlaced image) as in figure 
7. Assume here that inter-field interpolation pixels are 
generated for a block "c". In this case, the sum of the absolute 
values of differences between a block "f " in a forward reference 
field indicated by a static motion vector "E" and a block "g" in 
a backward reference field indicated by a motion vector M E' " that 
is in the opposite direction to the static motion vector "E" is 
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calculated. Here, the opposite motion vector "E ,n is a motion 
vector having the opposite direction to the motion vector "E" and 
indicating a reference field which has the opposite positional 
(forward or backward) relationship to the reference field 
indicated by the motion vector "E" with respect to the target field. 
The calculation of the sum of the absolute values of the differences 
between blocks means that differences between corresponding 
pixels which are coupled by arrows in the blocks " f" and "g" are 
obtained, and then the sum of the absolute values of these 
differences is obtained in block unit. Then, the obtained sum of 
the absolute values of the differences is used as the evaluation 
scale. In figure 18, only the vertical pixels are shown, while 
horizontal pixels are also targets for the calculation of the sum 
of the absolute values of differences between pixels as the first 
evaluation scale. 

Finally, in step S1103, an interpolation method for 
obtaining the minimum evaluation scale among the first evaluation 
scales obtained using the motion vector and the static motion 
vector in the above-mentioned processes is selected, and then the 
inter-field interpolation pixels are generated. 

Next, an example of the operation of the inter-field 
interpolation pixel generator 604 is described with reference to 
figure 12. 

In figure 12, since the operations in step S1001 to S1007 
are the same as those in figure 10, the descriptions are not given 



here. In step S1201, it is judged whether or not an effective 
motion vector (forward, backward, or two directions) includes a 
static motion vector. When the judgement result in step S12 01 is 
"No", the operation proceeds to step S1203. When the result is 
"Yes", the operation proceeds to step S1202. 

When in step S1202 the second evaluation scale is obtained 
using the static motion vector, the evaluation scale can be 
obtained similarly to the methods which has been described with 
reference to figures 14, 15 and 16. 

Finally in step S1203 , an interpolation method for obtaining 
the minimum evaluation scale among the second evaluation scales 
obtained using the motion vector and the static motion vector in 
the above-mentioned processes is selected, and then the 
inter-field interpolation pixels are generated. 

The inter-field interpolation pixels generated as described 
above are outputted to the weighting factor decision unit 606 and 
the progressive image generator 607. 

Next, the operation of the intra-field interpolation pixel 
generator 605 will be described. The intra-field interpolation 
pixel generator 605 generates intra-field interpolation pixels 
using pixels in the deinterlacing target field. 

A first example of the operation of the intra-field 
interpolation pixel generator 605 is described with reference to 
figure 19. Figure 19 is a diagram schematically showing a state 
of pixels in a deinterlacing target field of a decoded image 
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( interlaced image). When the intra-field interpolation pixels 
are to be generated for a block "c" in figure 19, the intra-field 
interpolation pixels are generated from the mean values of upper 
and lower pixels. For example, an interpolation pixel "h" is 
generated as the mean value of pixels "i" and "j" . 

A second example of the operation of the intra-field 
interpolation pixel generator 605 is described with reference to 
figures 20 and 21. Figure 20 is a diagram illustrating pixels in 
a deinterlacing target field, showing a case where an 
interpolation pixel "h" is generated. Figure 21 is a flowchart 
for explaining the operation of the intra-field interpolation 
pixel generator 605. 

Initially in step S2101, it is judged whether or not the 
absolute value of a difference between pixels in the vertical 
direction (pixels "i" and "q" ) is equal to or larger than a 
threshold. When the judgement result in step S2101 is "Yes", the 
operation proceeds to step S2102. When the result is "No", the 
operation proceeds to step S2106. 

In step S2102, the direction of an edge passing through the 
pixel "h" is detected using pixels surrounding the position of 
the interpolation pixel (pixel "h" ). This can be achieved by 
selecting a direction in which the absolute value of a difference 
between two pixels in the direction of the edge passing through 
the pixel "h" is the minimum value. More specifically, the 
absolute values of differences between two pixels in five 
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directions shown by arrows in figure 20 are calculated. Assume 
here that the absolute value of the difference between pixels "n" 
and "o" is the minimum among the absolute values of the differences 
in the five directions. In this case, a provisional interpolation 
pixel is generated from the mean value of the pixels "n" and "o M . 

Next, in the processes in steps S2103 and S2104, the 
strongness of the correlation between pixels which exist in the 
direction of the edge is obtained as the reliability of the edge. 

In step S2103, it is judged whether or not the ratio of the 
absolute value of a difference of pixels in the direction of edge 
(pixels "n" and "o !l ) to the absolute value of a difference of pixels 
in the vertical direction (pixels "i" and "q") is smaller than 
a threshold. When the judgement result in step S2103 is "Yes", 
the operation proceeds to step S2104. When the result is "No", 
the operation proceeds to step S2106. 

In step S2104, it is judged whether the value of the 
provisional interpolation pixel generated in step S2102 is a value 
between pixels in the vertical direction (pixels "i" and "q") 
values. When the judgement result in step S2104 is "Yes", the 
operation proceeds to step S2105, When the result is "No", the 
operation proceeds to step S2106. As described above, in step 
S2105, the provisional interpolation pixel generated in step S2102 
is adopted as the interpolation pixel "h". On the other hand, in 
step S2106, the interpolation pixel "h" is generated using the 
upper and lower pixels in the vertical direction (pixels "i" and 
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»q"). 

As described above, the intra-f ield interpolation generator 

605 performs the above-mentioned operation for all interpolation 
positions in the block, and generate the intra-f ield interpolation 
pixels. Then, the generated intra-f ield interpolation pixels are 
outputted to the inter-field interpolation pixel generator 604, 
the weighting factor decision unit 606 and the progressive image 
generator 607. 

Next, the operation of the weighting factor decision unit 

606 will be described. The weighting factor decision unit 606 
decides a weighting factor that is used when a weighted sum of 
the inter-field interpolation pixel and the intra-field 
interpolation pixel is obtained by the progressive image generator 
607. Here, the decision of the weighting factor is made in block 
units . 

Figure 22 is a flowchart for explaining the operation of 
the weighting factor decision unit 606. Initially in step S2201, 
the evaluation scale is calculated using the inter-field 
interpolation pixel inputted from the inter-field interpolation 
pixel generator 604 and the intra-field interpolation pixel 
inputted from the intra-field interpolation pixel generator 605. 
As the evaluation scale, for example the sum of the absolute values 
of differences between corresponding pixels, the maximum value 
of the absolute values of the differences, or the ratio of the 
mean values of pixels in a block is conceivable. Here, the case 
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where the sum of the absolute values of the differences between 
corresponding pixels is used as the evaluation scale will be 
described . 

In steps S2202 and S2203, comparison between the evaluation 
scale and a threshold is performed. When the evaluation scale (the 
sum of the absolute values of the differences between inter-field 
interpolation pixels and intra-field interpolation pixels) is 
smaller than a threshold THl , the weighting factor "w" is set at 
"1.0" (step S2206). When the evaluation scale is equal to or 
larger than the threshold THl and smaller than a threshold TH2, 
the weighting factor "w" is set at "0.5" (step S2205). When the 
evaluation scale is equal to or larger than the threshold TH2 , 
the weighting factor "w" is set at "0.0" (step S2204). Here, the 
relationship between THl and TH2 is that THKTH2. Then, the 
decided weighting factor is outputted to the progressive image 
generator 607. The weighting factor "w" here indicates a weight 
on intra-field interpolation. 

The progressive image generator 607 receives the inter- 
field interpolation pixel block from the inter-field 
interpolation pixel generator 604, the intra-field interpolation 
pixel block from the intra-field interpolation pixel generator 
605, and the weighting factor "w" from the weighting factor 
decision unit 606. Then, the sum of a value obtained by 
multiplying each pixel of the inter-field interpolation pixels 
by the weighting factor "w", and a value obtained by multiplying 
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each pixel of the intra-field interpolation pixels by "1.0- 
weighting factor w" is obtained as the interpolation pixel. Then, 
the decoded image (interlaced image) inputted from the image 
memory 507 is interpolated using the interpolation pixels, and 
a progressive image is generated and outputted. 

As described above, according to this third embodiment, in 
the case where a code sequence that is obtained by coding an 
interlaced image using motion compensation is decoded and the 
decoded interlaced image is subjected to the deinterlacing into 
a progressive image, the motion vector converter 602 converts a 
motion vector for the motion compensation, which is obtained at 
the decoding, into a size in one field unit, the motion vector 
judgement unit 603 judges the effectiveness of the motion vector 
which has been converted in one field unit, and the inter-field 
interpolation pixel generator 604 obtains pixels from a reference 
field using the motion vector which has been converted into the 
size in one field unit and the judgement result of the effectiveness 
of the motion vector, thereby generating inter-field 
interpolation pixels for a deinterlacing target f ield« Therefore, 
more accurate interpolation pixels can be obtained from the 
closest field on the basis of the motion vector which has been 
converted into the size in one field unit, thereby improving the 
quality of the image which has been subjected to the deinterlacing. 

In this third embodiment, the progressive image generator 
607 obtains the weighted mean of the inter-field interpolation 
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pixel and the intra-f ield interpolation pixel using the weighting 
factor decided by the weighting factor decision unit 606 to 
generate the final interpolation pixel. Therefore, the risk of 
generating an erroneous interpolation pixel can be minimized. 

Further, in this third embodiment, in a case where there 
are plural effective motion vectors when the inter-field 
interpolation pixel generator 6 04 generates the inter-field 
interpolation pixel, the inter-field interpolation pixel 
generator 604 calculates the evaluation scale for selecting a 
motion vector to be used for generating the inter-field 
interpolation pixel, and selects an optimum motion vector for 
generating the inter-field interpolation pixel with using the 
obtained evaluation scale. Therefore, the motion vector which is 
more suitable for the deinterlacing can be used and the quality 
of the image which has been subjected to the deinterlacing can 
be further improved. 

In this third embodiment, when the intra-f ield 
interpolation pixel generator 605 generates the intra-field 
interpolation pixel, it detects the direction of an edge in an 
interpolation position where the interpolation pixel is to be 
generated using pixels surrounding the interpolation position, 
and judges the reliability of the detected edge. When the 
reliability of the edge is equal to or larger than a predetermined 
value, the interpolation pixel is generated with using pixels 
existing in the direction of the edge. On the other hand, when 
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the reliability of the edge is smaller than the predetermined value, 
the interpolation pixel is generated with using pixels existing 
in the upper and lower directions of the interpolation position. 
Therefore, the image quality of parts in the image which has been 
subjected to the deinterlacing, notably oblique lines and the like, 
can be improved. 

Here, in this third embodiment, a code sequence of an 
interlaced image (MPEG video) which is coded by MPEG method is 
inputted, while an inputted code sequence may be coded by other 
method as long as an interlaced image is predictive coded with 
using motion compensation. 

In this third embodiment, the motion vector converter 602 
converts the motion vector into a motion vector in one field unit, 
while the motion vector may be converted into a motion vector in 
one frame unit. 

Further, according to this third embodiment, the motion 
vector converter 6 02 converts the motion vector into a motion 
vector in one field unit, while at that time the quantity of 
vertical motion may be restricted to the quantity of even pixels. 
In this case, the number of motion vectors which are judged invalid 
by the motion vector judgement unit 603 is reduced, thereby further 
improving the image quality. 

Further, in this third embodiment, the effectiveness of the 
motion vector is judged using the motion vector judgement unit 
603, while all motion vectors may be decided effective without 
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using the motion vector judgement unit 603. 

Further, in this third embodiment, the motion vector 
judgement unit 603 judges whether the size of the motion vector 
converted by the motion vector converter 602 is smaller than a 
threshold, and judges whether the quantity of vertical motion is 
an even number. However, the judgement as to whether the size of 
the motion vector is smaller than the threshold may be made using 
a motion vector value before converted by the motion vector 
conversion unit 602. 

In this third embodiment, as the method for calculating an 
evaluation scale for an obtained motion vector when the 
inter-field interpolation pixel generator 604 generates 
inter-field interpolation pixels, the first evaluation scale 
calculation method by which the sum of the absolute values of 
differences between pixels of a reference field indicated by an 
effective motion vector which is judged effective by the motion 
vector judgement unit 603, and pixels of a reference field 
indicated by a motion vector which is in the opposite direction 
to the effective motion vector is used as the evaluation scale, 
and the second evaluation scale calculation method by which the 
sum of the absolute values of differences between pixels of a 
reference field indicated by an effective motion vector which is 
judged effective by the motion vector judgement unit 603 and 
interpolation pixels generated by the intra-field interpolation 
pixel generator 605 is used as the evaluation scale have been 
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described. However, other parameter such as the maximum value of 
the absolute values of differences between corresponding pixels 
and the ratio of the mean values of pixels in a block may be used 
as the evaluation scale. 

Further, in this third embodiment, when the inter-field 
interpolation pixel generator 604 generates inter-field 
interpolation pixels, the processing is carried out in units 
(blocks) each being composed of 8 pixels (horizontal) x 4 pixels 
(vertical). However, the size of the block is not restricted to 
this . 

In this third embodiment, when the inter-field 
interpolation pixel generator 604 obtains the evaluation scale 
for the motion vector, the evaluation scales with respect to one 
direction, the opposite direction and the two directions of the 
motion vector are calculated. However, without calculating all 
of these evaluation scales, part of them may be calculated. 

Further, in this third embodiment, when the intra-field 
interpolation pixel generator 605 obtains the direction of an edge, 
the absolute value of differences between two pixels in the 
direction of the edge is used, while differences may be calculated 
using peripheral pixels . For example when an edge in the direction 
of pixels ,, m ,, -"p ,, in figure 20 is detected, the sum of the absolute 
values of respective differences of between corresponding pixels 
"i" and "o", pixels "m" and "p", and pixel "n M and "q" may be used. 
When the peripheral pixels are used for the calculation of the 
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differences, an erroneous detection of the edge can be further 
prevented for an image including noises or an image having quite 
fine patterns. 

Further, in this third embodiment, the intra-field 
interpolation pixel generator 605 generates interpolation pixels 
from two pixels, i.e., the number of taps in the filter at the 
time of interpolation pixel generation is two, while the number 
of the taps may be other values . 

Further, in this third embodiment, the intra-field 
interpolation pixel generator 6 05 finds the direction of an edge 
from five directions, while the number of directions may be other 
values . 

Further, in this third embodiment, the weighting factor "w" 
that is decided by the weighting factor decision unit 606 has three 
levels (1, 0.5, 0), while the weighting factor "w" may have any 
number of levels. 

Further, in this third embodiment, the weighting factor 
decision unit 606 uses the sum of the absolute values of differences 
between the inter-field interpolation pixels and the intra-field 
interpolation pixels as the evaluation scale. However, the 
evaluation scale may be obtained by other methods, or may be the 
maximum value of the absolute values of differences, the ratio 
of the mean values of pixels or the like. Further, the combination 
of these values may be used as the evaluation scale. 

Furthermore, in this third embodiment, a decoded image is 
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interpolated using interpolation pixels which are generated from 
inter-field interpolation pixels generated by the inter-field 
interpolation pixel generator 604 and intra-field interpolation 
pixels generated by the intra-field interpolation pixel generator 
605 , thereby generating a progressive image. However, a decoded 
image may be interpolated using only intra-field interpolation 
pixels generated by the intra-field interpolation pixel generator 
605, thereby generating a progressive image. In this case, in the 
deinterlacing apparatus 508, only the intra-field interpolation 
pixel generator 605 and the progressive image generator 607 
operate . 
[Embodiment 4] 

Hereinafter, a fourth embodiment of the present invention 
will be described. 

In this fourth embodiment, a description will be given of 
a case where a deinterlacing target macroblock that is to be 
subjected to the deinterlacing is an intra-coded macroblock. As 
the intra-coded macroblock, a case where a macroblock in a frame 
of a P or B picture is intra-coded and a case where a frame itself 
is intra-coded ( intra-f rame) are conceivable. Since the 
intra-coded macroblock is usually not accompanied with a motion 
vector, no motion vector can be used for the deinterlacing. Thus, 
in this fourth embodiment, when a deinterlacing target macroblock 
in a frame of a P or B picture is intra-coded, motion vectors of 
surrounding macroblocks, or forward and backward frames are used 
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for the deinterlacing. when a frame itself is intra-coded 
(intra-frame) , motion vectors of forward and backward frames are 
used. The difference of this fourth embodiment from the third 
embodiment is the operation of the motion vector converter 602. 
Thus , the operation of the motion vector converter 602 in the fourth 
embodiment will be described with reference to figure 23. 

Figure 2 3 is a diagram schematically showing a deinterlacing 
target frame including an intra-coded macroblock and forward and 
backward frames . 

The motion vector converter 602 reads a macroblock type of 
the deinterlacing target macroblock from the parameter memory 601 . 
When it is known from the macroblock type that the deinterlacing 
target macroblock is intra-coded , a motion vector of other 
macroblock is obtained. Here, other macroblock indicates a 
peripheral macroblock in the same frame as the deinterlacing 
target macroblock, or a macroblock in a forward or backward frame, 
which is at the same position as that of the target macroblock. 
In figure 23, macroblocks T, U, V and W are peripheral macroblocks 
in the same frame. A macroblock X is a macroblock at the same 
position in the forward frame. A macroblock Y is a macroblock at 
the same position in the backward frame. When the deinterlacing 
target macroblock is subjected to the deinterlacing, a motion 
vector of one of these macroblocks or motion vectors of plural 
macroblocks can be used. When the deinterlacing target frame 
itself is intra-coded, the motion vector of a macroblock in a 
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forward or backward frame, which is at the same position as that 
of the deinterlacing target macroblock is used. 

The motion vector converter 602 converts the size of the 
motion vector into a size in one field unit, and outputs the 
converted motion vector to the motion vector judgement unit 603. 
The conversion process of the motion vector converter 602 and 
processes subsequent to the motion vector judgement unit 603 are 
the same as in the third embodiment* 

As described above, in this fourth embodiment, in a case 
where a code sequence which is obtained by coding an interlaced 
image using motion compensation is decoded and a decoded 
interlaced image is subjected to the deinterlacing to be converted 
into a progressive image, when a deinterlacing target macroblock 
is intra-coded, the motion vector converter 602 converts a motion 
vector of a peripheral macroblock in the same frame or a macroblock 
in a forward or backward frame at the same position as that of 
the target macroblock into a size in one field unit, the motion 
vector judgement unit 6 03 judges the effectiveness of the motion 
vector which has been converted into the size in one field unit, 
then the inter-field interpolation pixel generator 604 obtains 
a pixel from a reference field using the motion vector which has 
been converted into the size in one field unit and the judgement 
result of the effectiveness of the motion vector, thereby 
generating an inter-field interpolation pixel for the 
deinterlacing target field. Therefore, even when the 
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deinterlacing target macroblock is an intra-coded macroblock, an 
inter-field interpolation pixel can be generated at high accuracy 
without the need for detection of the motion. 
[Embodiment 5] 

A fifth embodiment of the present invention will be described 
with reference to figure 24. 

In this fifth embodiment , the description is given of 
deinterlacing in a case where an MPEG video code sequence inputted 
by the MPEG video decoder 501 is a code sequence which has been 
read from a recording medium in a fast-forward or fast-rewind mode. 
Usually, when an MPEG stream is read in the fast-forward or 
fast-rewind mode from a recording medium which contains the MPEG 
stream, only I pictures are read out or only I and P pictures are 
read out. To be more specific, when only I pictures are read, the 
inputted code sequence is a code sequence including only I picture 
data, and when only I and P pictures are read, the inputted code 
sequence is a code sequence including only I and P picture data. 
Thus, when the MPEG video code sequence inputted by the MPEG video 
decoder 501 is a code sequence which has been read from the 
recording medium in the fast-forward or fast-rewind mode, the 
deinterlacer 508 cannot obtain a pixel from an adjacent field using 
a motion vector and generate an inter-field interpolation pixel, 
to perform the deinterlacing. Therefore, in this fifth embodiment , 
when the deinterlacer 508 receives reproduction method 
information indicating that the MPEG video code sequence inputted 
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by the MPEG video decoder 501 is a code sequence which has been 
read from the recording medium in the fast-forward or fast-rewind 
mode, it generates a progressive image using only an intra-field 
interpolation pixel . That is, the motion vector converter 602 
reads no motion vector from the parameter memory 601. Thus, the 
motion vector judgement unit 603 and the inter-field interpolation 
pixel generator 604 does not carry out any processing. Further, 
the weighting factor decision unit 606 outputs the weighting 
factor "0" to the progressive image generator 607 (i.e., an 
intra-field interpolation pixel is used as it is as an 
interpolation pixel) . Then, the progressive image generator 607 
uses the intra-field interpolation pixel outputted from the 
intra-field interpolation pixel generator 605 as it is as the 
interpolation pixel, thereby generating a progressive image. 

As described above, in this fifth embodiment, when an MPEG 
video code sequence inputted by the MPEG video decoder 501 is a 
code sequence which has been read from a recording medium in the 
fast-forward or fast-rewind mode, the progressive image generator 
607 generates an interpolation pixel only from an intra-field 
interpolation pixel generated by the intra-field interpolation 
pixel generator 605, and uses the interpolation pixel to generate 
a progressive image. Therefore, even when the inputted code 
sequence is a code sequence which has been read from the recording 
medium in the fast-forward or fast-rewind mode, the deinterlacing 
process can be performed without requiring a new component and 
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enlarging the circuit scale. In the case of fast-forward or 
fast-rewind mode, reproduced images are changed at high speeds, 
whereby the degradation of vertical resolution is not conspicuous 
even when the interpolation pixel is generated only with the 
intra-field interpolation pixel. 

As described above, according to the deinterlacing method 
and the deinterlacing apparatus of the present invention, when 
an interlaced image is to be converted into a progressive image, 
pixels of a deinterlacing target field and forward and backward 
fields are subjected to the VT filtering, thereby generating an 
interpolation pixel. At that time, the sum of the absolute values 
of differences between pixels of a deinterlacing target field or 
frame including the target field, and the immediately preceding 
and immediately following fields or frames is obtained, and the 
coefficient of the VT filter is decided on the basis of the sum 
of the absolute values. The coefficient is decided so that the 
image is judged to be a moving picture when the sum of the absolute 
values of the differences is larger, and the contribution (gain) 
from the adjacent fields is reduced. Therefore, in the case of 
still picture, a progressive image of high resolution can be 
obtained like in the case of using the prior art VT filter, and 
in the case of moving picture, the image quality of a part where 
the image quality is degraded by the prior art VT filter can be 
greatly improved. Further, these operations can be implemented 
by one filter, whereby the costs can be reduced. 
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According to the deinterlacing method and the deinterlacing 
apparatus of the present invention, when an interlaced image is 
to be converted into a progressive image, pixels of a deinterlacing 
target field and forward and backward fields are subjected to the 
VT filtering, thereby generating an interpolation pixel. At that 
time, the absolute value of difference between pixels of adjacent 
fields in the vicinity of a position to be interpolated is 
calculated, and the coefficient of the VT filter is decided on 
the basis of the absolute value. The coefficient is decided so 
that the image is judged to be a moving picture when the absolute 
value of the difference is larger, and then the contribution from 
the adjacent fields is reduced. Therefore, in the case of still 
picture, a progressive image of high resolution can be obtained 
like in the case of using the prior art VT filter. Beside, when 
there is a moving object or the like in the screen, the image quality 
degradation that is caused by the prior art VT filter can be 
prevented for that object. Further, these operations can be 
implemented by one filter, whereby the costs can be reduced. 

According to the deinterlacing method and the deinterlacing 
apparatus of the present invention, a code sequence which is 
obtained by coding an interlaced image using motion compensation 
is decoded, a decoded image of the interlaced image and a motion 
vector at the motion compensation are obtained, the motion vector 
is converted into a size in one field unit, the effectiveness of 
the motion vector which has been converted into the size in one 
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field unit is judged, and a pixel is obtained from a reference 
field on the basis of the motion vector and the judgement result 
of the motion vector to generate an inter-field interpolation 
pixel for the deinterlacing target field, an intra-field 
interpolation pixel is generated using a pixel in the 
deinterlacing target field, the weighted mean of the inter-field 
interpolation pixel and the intra-field interpolation pixel is 
obtained with a weighting factor to generate an interpolation 
pixel, and the decoded image is interpolated using the 
interpolation pixel, thereby generating a progressive image * 
Therefore, a more accurate interpolation pixel can be obtained 
from the closest field on the basis of the motion vector which 
has been converted into the size in one field unit, thereby 
improving the quality of the image which has been subjected to 
the deinterlacing. In addition, the final interpolation pixel is 
generated from the weighted mean of the inter-field interpolation 
pixel and the intra-field interpolation pixel, and further the 
weight is decided from the difference between the inter-field 
interpolation pixel and the intra-field interpolation pixel, 
whereby the risk of generating an erroneous interpolation pixel 
can be minimized. 

Further, according to the deinterlacing method and the 
deinterlacing apparatus of the present invention, in a case where 
there are plural effective motion vectors which can be used for 
generating the inter-field interpolation pixel, an evaluation 



scale for selecting a motion vector to be used for generating the 
inter-field interpolation pixel is calculated, and an optimum 
motion vector for the field interpolation process is selected with 
using the obtained evaluation scale. Therefore, the motion vector 
which is more suitable for the deinterlacing can be used and the 
quality of the image which has been subjected to the deinterlacing 
can be further improved. 

Further, according to the deinterlacing method and the 
deinterlacing apparatus of the present invention, when an 
intra-field interpolation pixel is generated, the direction of 
an edge in an interpolation position where an interpolation pixel 
is to be generated is detected using peripheral pixels, and the 
reliability of the detected edge is judged. When the reliability 
of the edge is equal to or larger than a predetermined value, the 
interpolation pixel is generated with using pixels existing in 
the direction of the edge, and when the reliability of the edge 
is smaller than the predetermined value, the interpolation pixel 
is generated with using pixels existing in the upper and lower 
directions of the interpolation position. Therefore, the image 
quality of parts in the image which has been subjected to the 
deinterlacing, notably oblique lines and the like, can be 
improved . 

Further, according to the deinterlacing method and the 
deinterlacing apparatus of the present invention, in a case where 
a code sequence which is obtained by coding an interlaced image 
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using motion compensation is decoded and a decoded image of the 
interlaced image and a motion vector at the motion compensation 
are obtained, when a deinterlacing target macroblock is 
intra-coded, a motion vector of a peripheral macroblock in the 
same frame or a macroblock in a forward or backward frame at the 
same position as that of the target macroblock is converted into 
a size in one field unit, the effectiveness of the motion vector 
which has been converted into the size in one field unit is judged, 
an inter-field interpolation pixel for the deinterlacing target 
field is generated from a pixel obtained from a reference field 
using the motion vector and the judgement result of the motion 
vector, an intra-field interpolation pixel is generated using a 
pixel in the deinterlacing target field, a weighted sum of the 
inter-field interpolation pixel and the intra-field interpolation 
pixel is obtained using a weighting factor, thereby generating 
an interpolation pixel, and the decoded image is interpolated 
using the interpolation pixel, to generate a progressive image. 
Therefore, even when an image is an intra-coded image, an 
inter-field interpolation pixel can be generated at high accuracy 
without the need of motion detection, thereby improving the 
quality of the image which has been subjected to the deinterlacing. 

Further, according to the deinterlacing method and the 
deinterlacing apparatus of the present invention, when an inputted 
code sequence is a code sequence which has been read from a 
recording medium in the fast-forward or fast-rewind mode, only 
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an intra-f ield interpolation pixel is generated, and a progressive 
image is generated using only the generated intra-field 
interpolation pixel. Therefore, a code sequence which has been 
read from the recording medium in the fast-forward or fast-rewind 
mode can be subjected to the deinterlacing without requiring a 
new component and enlarging the circuit scale. 



